Last few years have witnessed impressive progress propelled by deep learning methods. However, one critical challenge faced by existing methods is to strike a sweet spot of deep model complexity and resulting SISR quality. This paper addresses this pain point by proposing a linearly-assembled pixel-adaptive regression network (LAPAR), which casts the direct LR to HR mapping learning into a linear coefficient regression task over a dictionary of multiple predefined filter bases.

Weaknesses: The dictionary used in reconstructing HR images is hand-crafted. Why can the filters in the dictionary not be learned as kernels in neural network and enjoy the benefit of end-to-end learning as many pure deep learning-based SISR method? In the experiment, when comparing with SOTA SISA methods, only x2 and x4 results are shown while x3 results are missing. The authors are recommended to provide x3 results as well. In addition, FALSR-C and FALSR-A in Table 2 used only DIV2K as the training set, while the training set of the proposed method are both DIV2K and Flickr2K, and thus the comparison here is not fair.

This submission proposes to do single image super-resolution using a network which produces coefficients for a fixed bank of Gaussian/DoG filters. The super-resolution results produce nearly SotA super-resolution PSNR while the proposed approach is 1-2 orders of magnitude more efficient than SotA. Reviewers liked the idea of incorporating a filter bank dictionary. While all of the reviewers felt that these weaknesses put the submission below the acceptance threshold, metareviewers felt that the authors' response adequately addressed each of these concerns. Please add comparisons with the SotA approaches (EDSR, RCAN, ESRGAN, ProSR) in terms of PSNR, efficiency (MultAdds), and parameter count.

Last few years have witnessed impressive progress propelled by deep learning methods. However, one critical challenge faced by existing methods is to strike a sweet spot of deep model complexity and resulting SISR quality. This paper addresses this pain point by proposing a linearly-assembled pixel-adaptive regression network (LAPAR), which casts the direct LR to HR mapping learning into a linear coefficient regression task over a dictionary of multiple predefined filter bases. Moreover, based on the same idea, LAPAR is extended to tackle other restoration tasks, e.g., image denoising and JPEG image deblocking, and again, yields strong performance.